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Attorney Docket No. 38195.69 

A VIDEOP HONE S IGN LANGUAG E V I DEO I NPUT DEVICE FOR CONVERSATION 
ASS I STANCE W -I-T-H--^^ DEVICE FOR 

CONV ERS A TI ON WITH SI G N LANGUA G E , AND A SIGN LANGUAGE 
INTERPRETATION SYSTEM USING THE SAME 

BACKGROUND OF THE INVENTION 

1 ^ Technical Field of _the Inve ntion 

The present invention relates to a video input device, 
a video input/output videophone sign language conversation 
assistance device and a sign language interpretation system 
including u sing the same which are used by a-deaf -mut e persons 
to have a sign language conversation using with sign languag e 
) E> Y-~waY"~^> i communications means such as a videophone, and in 
particular , the present invention relates to a video input 
device for sign language conversat io n — w^h--s^B---^-a^€fu-a-g-e-7 — et 
video — input /output — device — §©f — sign — language — conversation 
videophone w ith - sign language conversation assistance device 
and a sign language interpretation system including using the 
same which are used to transmit p ref er able — #er — exampl e — 
t r an s mit ti n g a video other than sign language while performing 
expl a nation by s ign language. 
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2. Description of the Related Background Art 

While sign language is important for communicating with 
m eans fo r comm unic ations to a deaf-mute person, the picture 
quality of prior art videophones was poor and not sufficient 
for a--e^nv^irsa-tri^B-~^irirh— sign language conversations between 
deaf-mute persons in remote locations. With recent 

advancements in Nowadays^ w~i-th — the — adva-nee-me n t irR — t-he 

communications technology, the picture quality of a videophone 
has been greatly improved. Accordingly, sign lan g uage 
conver sat ion , thug allo w irn^--a"eoBve-3r^artion-wd th s-i-g-n-l a n g u age 
between deaf-mute persons in remote locations is now practical 
3lhc1 to be available stt — a practical level . 

Fig. 1 3 shows a conceptual diagram of a sign language 
conversation w-irt-h — s-i-gn — l-a-ngu-age — between deaf-mute persons 
u s i ng by— w-a-y— o^ a prior art videophone. In Fig. 13, a numeral 
10 represents a videophone terminal used by a deaf-mute person 
A and numeral 2 0 represents a videophone terminal used by a 
deaf-mute person B. The deaf-mute person A sets the videophone 
terminal 10 such so that his/her sign language will be 
capturedp-i-eked — up by an imaging section 1 Ob and the sign 
language of the deaf-mute person B displayed in a video display 
section 1 0a will be viewed. Similarly, the deaf-mute person 
B sets the videophone terminal 20 such-s-o that his/her sign 
language will be capturedpi cked up by an imaging section 20b 
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and the sign language of the deaf-mute person A displayed in 
a video display section 20a will be viewed. By doing so, the 
deaf-mute person A and the deaf-mute person B have a sign 
language conversation w-i-th si g n la ngu a g e via a videophone. 
While a cellular phone -ty-pe-is used as a videophone terminal 
in this example, a desktop- type videophone terminal may be also 
used. 

Next, a situation e a-s e will be described in whichw here 
a deaf-mute person converses with a non-deaf -mute person by 
using a videophone terminal via a sign language interpreter. 
Such sign language interpretation is implemented by using, for 
example, a multipoint connection unit which interconnects 
three or more videophone terminals to provide a-teleconf erence 
services. 

Fig. 14 is a conceptual diagram of a sign language 
interpretation service using a prior art multipoint connection 
unit. In Fig. 14, Mumeral 1 0 represent s a videophone terminal 
for deaf-mute persons used by a deaf-mute person A (hereinafter 
referred to as a deaf-mute person terminal), numeral 2 0 
represents a videophone terminal for non-deaf -mute persons 
used by a non-deaf -mute person B (hereinafter referred to as 
a non-deaf -mute person terminal), and numeral 30 represents 
a videophone terminal for sign language interpreters used by 
a sign language interpreter C (hereinafter referred to as a 
sign language interpreter terminal) . Numeral A — num e r al — 1 
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represents a multipoint connection unit. 

The multipoint connection unit 1 accepts connections 
from the terminals 1 0, 20, 30, receives video and audio 
transmitted from the terminals, synthesizes the received video 
and audio, and delivers the resulting video and audio to each 
terminal. Thus, a video obtained by synthesizing the videos 
from the terminal is displayed on the display screens (10a, 
20b, 30b) of the terminals. An audio obtained by synthesizing 
audios collected by the microphones of the headsets (20c, 30c) 
and fc far c like is output to loudspeakers such as the headsets 
(20c, 30c) of the terminals. Synthesis of videos uses^ for 
example L a four-way synthesis which equally synthesizes the 
videos of all parties engaged. The deaf-mute person A does 
not use audio input/output such^e that the headset of the 
deaf-mute person terminal 10 is omitted and voice 
communications are provided only between the non-deaf -mute 
person and the sign language interpreter. Where -i~H-~earS-e the 
environment al or background sound is collected by the deaf-mute 
person terminal 10 and transmitted, or where i~n~-ea-s-e a helper 
is present with the deaf-mute person , a microphone or a headset 
may be provided. 

With this configuration, when the deaf-mute person A 
performs sign language, the sign language interpreter C watches 
the sign language of the deaf-mute person A and translates it 
into a voice. The non-deaf -mute person B listens to the voice 
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of the sign language interpreter C to understand the sign 
language of the deaf-mute person A. When the non-deaf -mute 
person B speaks, the sign language interpreter C listens to 
the voice of the non-deaf -mute person B and translates it into 
sign language. The deaf-mute person A watches the sign 
language of the sign language interpreter C to understand the 
speech of the non- deaf -mute person B. 

However, in a conversation between deaf-mute persons 
using a videophone or a conversation between a deaf-mute person 
and a non-deaf -mute person via sign language interpretation, 
the videophone terminal for deaf-mute persons must cap turep-ie-k- 
up the sign language of the deaf-mute person and transmit the 
video to the distant party while the deaf-mute person is 
performing sign language, such -so-that the videophone terminal 
for deaf-mute persons cannot transmit other videos to the 
distant party. Thus, the deaf-mute person cannot transmit a 
video other than sign language and explain wh-jrl-e-e-x p l ain i ng - the 
video using b-y sign language in a videophone conversation. 

In this mannerway, while it is possible to transmit a 
target video and exp 1 ain wh-il^—e-x-pl-a-irn-i-Hg the video using -by 
wa-y-e#-a-voice in a videophone conversation between unimpaired 
persons, it is not possible to transmit a ^t-h er e is no- wa-y— o-£ 
-tra^sfTHrt^i-ng--a-- target video and e xp l_a inv^ i i 1 c e-x^l-e-i-n-i-ng the 
target same video in a videophone conversation involving a 
deaf-mute person. As a result, explanation of a^-he target is 
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imprecise and er^a Ghort quick spccdy conversation is difficult . 



SUMMARY OF THE INVENTION 

To overcome the problems described above, preferred 

embodiment sA -ma- i n o b ject of the present invention irs-t-e-provide 
a videophone sign language conversation assistance v-i-dee-i-npti-t 

device — far e-onver satire?*- — w-i-t-h s-i-g-n — l-a-ngu-age T a — video 

^.p.^/^.^ — .fi0.jp — ^^.3.^.3^^^ — w-i-t-h — s-i-g-n — 1-a-nguage- 
i n cludin g the sam e, and a sign language interpretation system 
using the videophone sign language conversation assistance 
device same to ena bl e-sa-me — enablin g a deaf-mute person to 
transmit a target video other than sign language while 
performing explanation by sign language. 

.According to a prefe r red embodiment of the present 
invention, a videophone sign language conversation assistance 
device used by a deaf-mute person to have a sing language 
conversation with — sign — language — w it h — tke — u-s-e — ab using a 
videophone includes hand imaging means including waist fixing 
means to be fixed at the wai st o f a deaf-mute pers on for 
capturing images of the hands of the deaf-mute person to acquire 
a sign language video, sight line direction imaging means fixed 
to the head of the deaf-mute person and arranged to c ap ture 
images of the area in the direct io n of the sight line of the 
^ !L§ ^ I m ^ ^ e _ E?£ s ° n j __ Yi d e o s i gnal synthesis means for 



synthesizing a^video^signal^acquired by the hand imaging means 
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and a video signal acquired by the sight line direction imaging 
means, and a videophone connection means including a function 
to transmit a video signal synthesized by the video signal 
synthesis means to a videophone terminal , wherein the deaf-mute 
person can include an explanation by sign language while 
transmitting a video in the sight line direction. . 

The in v en- tion d e scrib e d in claim 1 is a v -* deo input d -ev-iee 

for co nv-e-p-s-a t i o n w ^&3~s4^-B-lra^gTO 

i magin g m ea ns for pick i n g up si gn languag e; v i deo a-equ-i-s^r-^-OR- 
means for acqu iri ng a video othe r than si gn la nguage; v ideo 
synthe sis means for sy nthe sizing a si gn language v i deo p icked 
up by th e sign language im a g i ng means and a vid e o other than 
sig n langu a g e acquir e d by the vi deo ac qui s-ir^irOf^-^ee-RS-; — a-nd 

a---~de a--f •-•mU"te~-pe-r^ 

v-i~deo~^%^e-r--^hba-n---s4^n- language- — 

With this configuration, the deaf-mute person can 
precisely explain the target in the sight line direction, and 
thus L a conversation with sign language is sped can be sp eeded 
up . 

The videophone^ connection means can be connected to a 
videopho ne o f t he cellular phone — type . Thus L a deaf-mute 
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person can transmit to the opponent party a video other than 
sign language with explanation by sign language added even 
while moving, which adds to the convenience to the deaf-mute 
person. 

The — The inv en-t ion de-se-r-ibed in ejra-i- m 2 is the vi-deo input 

4__ — ehafaefewiz-ed — i~n — t-ha-t— t-he — s-ig-n — l~ang-uage — imaging — means- 
eompr4.~ses-~wa-i-s-t f ix -i-ng—mean-s-- te— be—f-irxed— at- t h c—w-a4-s4--e^-----the- 
dcaf mut e p erson. 

Wit h th is c on f i gu rati on, t he-sign language of the 

deaf-mute person is picked up captured und er c ertain conditions 
up-and transmitted to the opponent party other end u n dr cr certain 
conditions — even when the deaf-mute person changes his/her 
position or orientation ._ This allows a 7 — thereb y — enablin g 
stable s-i- gn languag e c onversation with sign language. - — 

The video signal synthesis means preferably includes a 

function to synthesize a video signal captured by the sight 
line direction imaging means as a main window and a video signal 
ac quired by the hand imaging means as a sub window in a 
Picture- in-Picture arrangement and a function to change the 
set tj,ng __of __the P°?ition of _ the sub_wjLndow 

The videophone sign language conversation assistance 
device p ref erably includes display me ans fix e d to the hea d o f 
the_ deaf-mute person for displ ay ing a video received by the 
videophone^tejrmjLnal^in front of the eyes of the deaf-mute person 
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and simultaneously allowing the deaf-mute person to view the 



outer world including a tar get for sig n lang u age conversation ; 
and The in v entio n d e scr -i-bed---i- n clai m 3 i-s — ^he— v-i-d e o in pu-t- 
device for con v ersation with sign langua ge accordi ng t o cl a i -m 

-1 — or — 2~, — eha-ra e--te-r-i-z-ed—i n t h-a^-^-he-— v^d^e---^a-n^in^^-i-e-n— me^-ns- 

eeHvp^i~ses~^4rdeephene™eenRee-t^ — t-o— be— eon-neet-ed— ^=>~a 

v-ideophene—terHyrn-a-l-: 

W-i-t^ 

a gen e ral - purpose vid eo ph o n e t erminal to transmit a-v-i-dee-et her 
th an sign l a ngu a ge w-i-t-h— e-xp 1 an-a-t-i-o n by sign 1 ang-u-ag-e--add-ed— 

In p articular; — t&e--v-id:eo phon e eonneetion mea-n- s can — be 

connected to a v id eophon e of th e cellul ar phone ty p-c s o t h at 
t-he — deaf mute per son c an transmit — a video — other — than — sign 
l anguage with explanat i on by sign languag e added while on the 
r oad ; — This a dds—to the-eeflveRienee to—t h e d eaf - mut e person-.- 

gihe inven^^n^eser4irbed-^n-~ e-l-a-i-m— 4— is t h e~v^^eo— i-npu^t 

dev-iee— for co nv e rsation wi t h sign la ^ig-u-age-a-c cord ing to clai m 

-3- r eharae-ter-i-zed in t~hat" the videophone connection means 

pr ef e r abl y includ e s a fun ction to re ceiv e a vi deo sign al from 
the videophone terminal and supply the video signal to the 
display means. 

With this configuration, th e deaf- mu te person is able 
to_ include an explanation by sign language while transmitting 

a „yi^?CL „ ot ' ier than s ig n language, as well as receive an 

exp 1 ana t i on by _sign 1 anguage whil e viewing the outer world by 
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freely shifting his/her sight line . T he di s play means fix ed 

i n front of the deaf-mute person ±s_ preferably as sma ll as 
possible so as not to obstruct viewing of the outer world. 

The sight line direction imaging means and the display 

means are preferably molded into a frame which can be fixed 
to the ears and nose of said deaf-mute person. 

This enables the deaf-mute person to readily set the 

si ght line direction imaging means and th e display means at 
the optimum position in front of his/her eyes. 

_ The_ videophone c onn ection means preferably 
includesee-mpr^i-is-e^s- radio communications means for performing 
radio communications with the videophone terminal. 

This eliminates the need With this c on figuration, 

i-t — is — R-e — l o ng e r — necessary to connect the videophone sign 
language y-j-deo--i-npu-t-dev-- ice f er~conver sation assistance device 

to wit-h si gn—lrarng-u a g e-w-i- t&~-a videophone terminal via a cable, 

which greatly facilitates sys-t-em— handling. 

According to another preferred embodiment of the present 

invention, a video p hone s ign language i nterpretat ion system 
connecting connects the videophone sign language conversation 
assistance device according to the preferred embodiment 
described above with the video-phone terminal of a deaf-mute 
person, and interco n n e cting — interconnects the videophone 
terminal of the deaf-mute person, the videophone terminal of 
a non-deaf -mute person and the videophone terminal of a sign 
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langua ge interpreter in order to provide sign languag e 



interpretation by a sign language interpreter in a videophone 
conversation between a deaf-mute person and a non-deaf -mute 
person, wherein the videophone sign language interpretation 
system includes terminal connection means including a sign 
language interpreter registration table where the terminal 
number of the videophone terminal of a sign language 
interpreter is registered, the terminal connecti on mea ns 
inc luding a f uric t i on to accept a call from the videophone 
terminal of t he deaf-mute person or videophone terminal _of _the 
non-deaf -mute person, a function to prompt a calling videophone 
terminal for which the call is accepted to enter the terminal 
number of the called terminal, a fu n c t ion to extract the 
te^inal number of the videophone^ terminal of a sign language 
interpreter from the sign language interpreter registration 
table, a function to call the videophone terminal of a sign 
language interpreter by using the extracted terminal number, 
and a function to call the called videophone terminal by using 
the_ acquired called termi nal number, _ and__ video/audio 
communications means including a function to synthesize at 
least a video from the videophone terminal of the non- deaf -mu t e 
person and a video from the videophone terminal of the sign 
language interpreter and transmit the resulting video to the 
videophone terminalof the deaf-mute person, a function to 
transmit at least a video from the videophone terminalof the 
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deaf-mute person and an audio from the videophone terminal of 



the sign language interpreter to the videophone terminal 
of the non-deaf -mute person and a function to transmit at least 
a video from the videophone terminal of the deaf-mute person 
and an audio from the videophone terminal of the non-deaf -mute 
person to the videophone terminal of the sign language 
interpreter . 

In this manner, a function is provided to extract and 

call the ^terminal jiumber of a sign language interpreter 
registered in_a sign language interpreter registration table. 
A sign language interpreter can provide sign language 
interpretation anywhere he/she may be, as long as he/she h as 
access to a videophone terminal This provides a flexible and 
ejfficient sign language interpret ati on jsjystem^ 

Selection information for selecting a sign language 

interpreter is preferably registered in the sign language 

interpreter registration table, and the terminal gi^e— 

i-nvett-M--en--d-e 

fe-r-eonv-e-r-s-a-t-i eempr-i-s-i-n^ the-v-idee i-n^u-t 

£ e . v 4.. ee .™§ e . F .-.. eeiWe .^ aeee-r^iiftf -to eia-i-m 

- 3 or 4 ; t h -e v idc-ep-hej^e-ee^fie ct ion -means i nc l-udi.-n^-s-ig^-4ra-n^u^ge 
■v-idre o re e eiviR ^- mcans for r e ceivin g a sign 1 a-n^fueg-e-vi-de-o beiftef 

video i nput/output d e v ic e for conversation with sig n language 
in c l u des disp lay means for displaying a sign languag e v ide o 
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received by the sign la n gua ge vide o rece iving means and fixi n g 
means for fixing the display means in front of the eyes of the 
d eaf mu-t-e—per son- 

With this configurati on — t-he—deaf mute person is ab k 

•fee-add expla nati o n b y sign languag e-w hile tra^s-mit ti-ng a v-ideo 
e-tfe^^-fe-haB— — a^~we4i---a-&--€fe-t— e-xpiaBe^€H^by— &ig-H 

h-is/her si-ght l-i-^ic . T-he-^-i-s^>lray"-me-a-n-5 f i-x-ed— irn--#3rO n t of t rhe 
eyes of deaf mut e person is preferably as s - mall as poss-irb-le 
s-e— as—no t t o—hampe^^v i o wing o j-the-e uter ^0^44^ — 

T4^e — i-B-v ent io n — d esc ribed — in — claim — 6 — i-s — the — video 

input /outp u t — device — £er — conversation — with — sign — l an gu a g e 
acc o rd ing to claim 5, characterized in th at the display means 
compr is es a co nv ex l ens w hich can sub s tantiall y fo cus on a sign 
languag e — vid e o — dis played — on — the — displ ay — m eans — when — the 
dea-f--mu-tre^er^so«-"Vi-ew-s — th:e™-o : u-trer^-w0-r-l-d-i — 

W-i-fe-h--£-h is con f -ig^u-ira-t-jro n , t h e-de a#— mu-te-pe r s o n-ea-n-wa-t-e-h 

a~--~ s i:gn~--~l^--ng-u^^ 

mak--i-ng---f-ee-u-s----a outer- 

wor-l-d - - The s ig n langu ag e • vide o di sp l ay ed- en t he d-i s-p lay - means 

dev-i-ee i s red-ueeeh 

input /out p ut — device — fe-r — conversation — with — sign — language 
accord i n g t o claim 5 or 6 , — charac teri z e d in that — the — fixi n g 
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means has a frame str uctu re which can be fixed to the cars and 
nose of a deaf mute p e rson. — 

W ith — th-i-s — con fi g uration the — deaf-mute — person — e-a-n 

r^ead-il y s et the disp lay-m eans at the o pti mum p ositio n-in front 

peweih — 

gi^e- — i-n-ve-nt-ien — de-se~r~i~bed — -i-n- — eia-i-m — -8 — i-s — t-he — -video 

according to any — one of claims 5 through 7, char a -e- terized in 
t ha t the v i de o ae<^-i-s^ti-e n-me-a-n s i n cludes-fe-a-^ g c t imag i n g m e-an-s 
•f er picking up a targ et other than si gn lang uage and the target 
im aging means is fixe d to t he fixin g means and picks up t he 
area in the v iewing direction of the deaf mute p erson. — 

With — th i s — co n fi guration, the — deaf mut e — person — e-a-n 

-t-ra-R-sffl-i t t he--~~t-a-r g e t vi-de-o--w-±-t-h----e-x-p-l a n a t i a n-%y--s4-g-n--l-a-ng-ua-ge 

-£e3r- Gxp lar i at ion. — This a llows th e dc a-f— -mu t e p e^e-s-en— to—ha v e a- 

mere speedy eonversat-ion with sign-— 1-angu-age in-a mere pree-ise 

•fa-s-h-i-OFK — 

-The i n ve n t i en --de s e"r i bed i n e l a i m- 9 i s a s ign l angu ag e 

u-s-i-ng — s-i-gn 1-a-nguage te — whireh — i-s — ee-nnee-t-ed — the — v-i-dee 

input/ output — device — for — conversation — w ith — sign — language 
accordi ng — te — a-ny — e-ne — eS — claims — § — t hr o u gh — 8-7 — a — vi d eop ho n e 
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ter m inal — for non deaf mute persons — us e d b y a non deaf mute 
person — incapable — of using — sign — language , — and — a videophone 
ter^mi-nal far sign-jrare guage int c -irp-re tcrs u s cd-by-a-s-i-gn—l-an^uage- 
int c rpr c tcr i n orde r to provide sign languag e int er pretati o n - 
-i-n a — e onvcrs -atien — between — a — d eaf m u t -e — p erson and — a 

i-n-t^rp-re-t-e-r re<^l^trarti~e**— tabjre-~wfae r e t he— t-e-rm-i-n-a-lr 

•number of the vi-deephene term i nal for sir^n lengurage 

i-nte- rpr ct crs us e d b y a si gn languag e intc rpr c t er^-^-r egistc re-d-r 
the-eeirae ctlon -^ft eans i nc i ting a fu nc tio n t e -ae cept a-ea4-l--fr-om 
the v ideopho ne -t ermin al f or deaf mute p e rs o n s o r th e v i d eo phon e 
t e rminal — for non deaf mut e p e rsons, — a function to prompt a 
calli ng v i d eo ph o n e ter minal for w hich t h e c all is accepted to 




#-r-efn — the — sign — 1-a-ng-u-a^e — i-nt erpr e-ter — r e g i str-at^-on — tabie 7 — a 
.f.. u ..^ e _^.Q^™™^. < = > .™.. < =i. s -l_^ : — ^-^.^..--^4^.3.0^^.0.^.^. — k^p^^-j^..^ — -§.0.jc — s-i-g-n — language 

in terpre ter s by -us i ng t he extraeted ter minal number of t he s ign 

-l~anguage — i-nte-rprete-rr - and — a- f^-netieR- — e ^i-l~ — t-he— e-a-l-led 

eeffl munications mean s— i n c l udi n g a f unet-i-on~to-s yn thcsi z e 

at least a video from t he videophone termi nal fo r non deaf mute 
pers o ns — and — a vi deo — from the videophon e — ter min al — for 5 — sign 
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language int erpreters a nd transmit the resulting video to the 
videophon e — terminal — for — d e af mute — persons , — a — function — t-e 
transmit — at — l east — a vide o fr o m t h e— v i d co-ph QRe—t-erflvi-narl — for 1 
deaf mute p e rs o n s and an aud i o fr o m t h e videophone t e rminal 

HeH--dea#-mH"te~-pe-r~s-eH5i — a ^. ( ^ a „4 TO . ( ^..3 :e ^4 6 -„t--ra-B-s-ffli t at l~ea-s-"t~--a 
■v^dee-f^r^ra— trhe^ 

a . H ._.. a .. H 4i 0 ._.f.. r . em .....t..he--vi-d f. e . F ..... n . 0 . R -_.^. e . a ..f__. m . u ..t~e"per-s 

to the videeph-ene~~t-erm inal for — sign langua -ge inter p-i^e-^e^s— 

^^kk--£j=HrS^ t i o n , t h e-de-af -mu t e p e r s o n c an h a v e 

a videophon e conversation with a non d e af mut e p er son - via si gf* 
l an g uag e i nt erpre t a t io n b y usin g t h e video input /output dcv jree 
for conv ersation w it h — sign language. — 

A s a fu n ctio n of thi s inven tion to extrac t and call the 

t er min alr-^ufflber o f a si gn l anguage interpr e t e r r e gistered i-n 

a — sign — 1-a-nguage — iftteirp^e^ter — e-a-n — pr e s e nt — a — s-ig-n — ta-nguage 

•ia-t-erp-ret-^ he^sfee-^a^-key- a .. s .....4 eB g:....... a . s . -he/she ea-n 

be earl-led— thereby a flexible and ef-f ieient sign language 

•interpretation system ea-n be p-Fevided- 

^e-irnve n t i on-de-s-e^irbed— in claim 10 i -s-t-he-s-i-^-n— lra-ng--uage 

jrn-t erpr e tation s y-s%eirv^-eeo^d-irng to cla im 9, ehar^ete^irzed--i-n 
-that — s-e-ieeti-e-n — i-n f o r-m-a-fe-i-OB — for — select i-ng — a — s-ig-n — la nguag e 
interpreter — is — registered in — the — sign — language — interpreter 
r e gistration table and that th e connection means includes a 
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function to acquire the conditions for selecting a sign 
language interpreter from the calling videophone terminal and 
a function to extract the terminal number of a sign language 
interpreter who satisfies the acquired selection conditions 
for the sign language interpreter from the sign language 
interpreter registration table. 

With this configuration, a sign language interpreter who 
satisfies the object of the conversation eve r a v i deo phone 
between a deaf-mute person and a non-deaf -mute person from 
among the sign language interpreters registered in the sign 
language interpreter registration table is c a n b e selected. 

The sign language interpreter registration table 
preferably includes an availability flag to register whether 
a registered sign language interpreter is available , and the 
control means has a function to rcf crenco pref erably refers to 
the ~ — T-he-eon-lHr o llc r referenc e s an availability flags in the 
sign language interpreter registration table to extract the 
terminal number of an available sign language interpreter. It 
is thus possible to automatically select an available sign 
lan gu age interprete r . This eliminates unnecessary -; — trhe-reby 
eiim-irna^-irng — u-se-lee-s calling and provide sp-rov-id-i-ng a more 
flexible and efficient sign language interpretation system. 

The t e r mi nal connection me an s preferably includes a 
function to regist_er_a term in the term registratio n t able via 
an o per ati on from a videophone terminal , a fun ction to select 
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a term to be used from the terms registered in the term 



registration table via an operation from a videophone terminal , 
a function to generate a telop of the selected term, and a 
function to synthesize the generated telop onto a video to be 
transmitted to the opponent party. 

This makes it possible to display, in a telop, on the 

videophone terminal of the conversation partner a term that 
is hard difficult to explain with sign language during sign 
1 anguage in te rpr etation_ or_ a word t hat is hard difficult to 
pro nounce . 

The above _ a nd other fea ture s , elements, step s, 

ch aract eristic s and advanta ges of the present i nvention will 
be apparent f rom the following detailed descr iption of 

E r ^€§^£§4. embodiments of t he inve ntion made referring to 

drawings . 



BRIEF DESCRIPTION OF THE DRAWINGS 

The above eb^eet-,- ether -efe-j-eet-Sr — e-ha-r-ae-t-e^i-s^-ire-s and 

advant age s e--f --the i-Hven tie 11 wi ll -be app ar ent f r om- the .f o-i-i-ew-ing 

ref er r i-ng— ^ o draw ings 

Brief Description of the Drawings 

Fig. 1 is a system block diagram of a video input/output 
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device — §e^ — conversation — with — sign — languagc v ideophone sign 
language conversation assistance device according to a 
pref erred a-n- embodiment of the present invention; 

Fig. 2 shows examples of a video displayed on the terminal 
of a t h e opp o nent " party of a conversation with sign language 
via by w ay— e-g the video input /output device for sign languag e 
conversation w-i-t-h — s-i-g-n — l-aH-g-u-age-— according to a preferred 
embodiment of t he present inve ntion; 

Fig. 3 is a system block diagram of a sign language 
interpretation system according to a pref erredm embodiment 
of the present invention; 

Fig. 4 shows an example of a video displayed on each screen 
of a deaf-mute person terminal, non-deaf -mute person terminal , 
and sign language interpreter terminal in sign language 
interpretation using the sign language interpretation system 
according to a preferred embodiment of the present invention; 

Fig. 5 is a processp ^eee-s-s-i-ng flowchart of a controller 
in a sign language interpretation system according to a 
pref err edan embodiment of the present invention; 

Fig. 6 shows an example of a sign language interpreter 
registration table ; 

Fig. 7 shows an example of a screen for prompting input 
of a called terminal number; 

Fig. 8 shows an example of a screen for prompting input 
of sign language interpreter selection conditions; 
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Fig. 9 shows an example of a screen for displaying a list 
of sign language interpreter candidates; 

Fig. 10 is a system block diagram of a sign language 
interpretation system according to another preferred 
embodiment of the present i nve ntion; 

Fig. 1 1 shows an example of a connection table; 

Fig. 1 2 is a processing flowchart of the connection 
processing of a sign language interpretation system according 
to another preferred embodiment of the pr e sent invention ; 

Fig. 13 is a conceptual diagram showing a conversation 
with sign language between deaf-mute persons by using a prior 
art videophone terminal; and 

Fig. 14 is a conceptual diagram of a sign language 
interpretation service using a prior art multipoint connection 
unit. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Bes*— Made— #e-r~~ Garayift^F— ^he~~-I-Rvent-i-en 

Fig. 1 is a system block diagram of a video input/output 

device i~en~ e . 0 . R .v e -F-sat"io.. n - with - s ign language 

convcrsationv ideophone sign language assistance device 
according to a preferred an-embodiment of the present invention. 
In Fig. 1, a—numeral 12 represents a display device for 

displaying a sign language video, numeral 13 represents a 

fixture for fixing the display device 12 in front of the eyes 
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of a deaf-mute person, numeral 14 represents a sign language 
imaging camera for picking up the sign language of the deaf-mute 
person, numeral 1 5 represents a waist fixture for fixing the 
sign language imaging camera 14 at the waist of the deaf-mute 
person, numeral 1 6 represents a target imaging camera, for 
picking up a target other than sign language, numeral 1 7 
represents a video synthesizer for synthesizing a video from 
the sign language imaging camera 14 and a video from the target 

imaging camera 16, nume ral 18 re p resen ts a videophone 

connection device for connecting the display device 1 2 and the 
video synthesizer 17 to a videophone terminal 10. 

The display device 12 uses, for example, a small-sized 
liquid crystal display having a sufficient resolution to 
display a sign language video. The display device 12 
enl argcs m agni f ie s a video such - &e-that a deaf-mute person can 
recognize the sign language displayed with the fixture 13 
attached. A convex lens is attached on G-n the surface of the 

display device 1 2 , such is attaehed a eon-vex- lens so that sign 

language displayed on the display device 1 2 is s-ubs^a-r^t-i-alrl-y- 
brought into substantial focus while the deaf-mute person is 
viewing the outer world, such as^ the conversation partner and 
the scenery. This enables a-l-l-e-w^ the deaf-mute person to 
easily no-^ma-l-ly- recognize the sign language displayed on the 
display device 12 while viewing the outer world. 

The fixture 13 includes has a spectacle frame structure 
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which can be fixed to the ears and nose of a deaf-mute person. 
Near the frame in front of the eyes of the deaf-mute person 
the display device 1 2 is attached 4rho di s-pl-ay— dev-±^e 1 2 for 
viewing of sign language without impairing the sight of the 
outer world. While the display device 1 2 is provided in a_lower 
left position location in front of the eyes of the deaf-mute 
person in this example, the display device 1 2 - i-t-may be provided 
anywhere as long as it does not impair the sight of the outer 
world. 

While the display devices 1 2 are provided on the same 
right and left positions locations of the fixture 1 3 such that 
the d eaf-mute p erson can so as to more clearly view recogniz e 
the displayed sign language in this example, the display unit 
12 may be provided on either side of the fixture 13 as long 
as the deaf-mute person can v i ewreeegn ize the displayed sign 
language . 

The fixture 1 3 is provided to sct used to locate -set— the 
display device 1 2 in front of the eyes of the deaf-mute person, 
such so that the display device 1 2 may be fixed to a hollow frame . 
Or, a transparent plate may be provided in a frame and the 
display unit 12 may be adhered s tuck- to the transparent plate. 
Whereto — ease the deaf-mute person has myopia, hyperopia, 
astigmatism, or presbyopia, and thus, requires — need-s- a 
corrective lens, a corrective lens may be provided in a frame 
and the display device 1 2 may be adher ed s t uck to the corrective 
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lens . 

The sign language imaging camera 14, such as which may 
be-a small-sized CCD camera^ is fixed to the waist fixture 15. 
The In this practice, — th e sign language imaging camera 14 is 
set to an angle of view that is wide enough to capture the image 
ofp-i-ek- up the sign language of the deaf-mute person while being 
it i s — fixed to the waist fixture 15. 

The waist fixture 15 is^ for example^ a belt to fix the 
imagin g c amera 14a at the waist of a deaf-mute person. Any 
waist fixture may be used which as long as th e waist fixture 
includes awhose buckle h avi ngha-s- an arm for fixing the sign 
language imaging camera 14 thereto so as to enableallo w the 
sign language imaging camera 14 to be set in an orientation 
such t hat wh crc the sign language of the deaf-mute person can 
^ e captured, p -i-ek-ed — up-r- This makes it possible to stably 
capture pi-e k up the sign language of the deaf-mute person by 
using the sign language imaging camera 1 4_^_ even when the 
deaf-mute person changes his/her position or orientation. 

The target imaging camera 16, such as -which- may — be 

s-i-m-i-l-ariy a small-sized CCD camera^ is fixed to the side of 
the fixture 13. When the deaf-mute person wears the fixture 
13, the azimuth of imaging by the target imaging camera 16 is 
substantially the same as the direction of the sight line of 
sight of the deaf-mute person. This precisely captures the 
target for conversation pr ec is el y for transmission of the video 
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obtained . 

The video synthesizer 17 synthesizes a target video from 
the target^ imaging camera 16 and the sign language video from 
the sign language imaging camera 14 into a single synthesized 
video. Several methods for synthesis shown in Fig. 2 are 
available . A ; a method may be selected therefrom depending 
on the purpose. Fig. 2(a) is a Picture-in-Picture 
representation where the target video is shown as a main window 
and the sign language video is shown as a sub window. On the 
other hand, Fig. 2(b) is a Picture-in-Picture representation 
where the sign language video is shown as a main window and 
the target video is shown as a sub window. Fig. 2(c) is a 
Picture-in-Picture representation where the target video and 
sign language videos are displayed in equal size. Fig. 2(d) 
shows the sign language video alone. Fig. 2 (e) shows the target 
video alone. Fig. 2(f) is a Picture-in-Picture representation 
where a still picture ofw-i-^-h the target video f roz-efl-is shown 
as a main window and the sign language video is shown as a sub 
window. On the other hand, Fig. 2(g) is a Picture-in-Picture 
representation where the sign language video is shown as a main 
window and a still picture ofwi-th the target video #r-o^eR~ is 
shown as a sub window. 

JFJl!L setting-~S-e4^t-i-ng of the position of the sub window 
in a Picture-to-Picture representation is preferably subject 
to change as required so as not to obstru ct the view ofm-ars-k 



24 



important information in a main window or hide another sub 
window inserted in sign language interpretation described 
later . 

The video synthesizer 1 7 may be accommodated in the waist 
fixture 1 5 or fixture 1 3 so as to supply a video signal from 
the target imaging camera 16 or sign language imaging camera 
1 4 to the video synthesizer 1 7 accommodated in the waist fixture 
15 or fixture 13 over a wired or wireless connection. 

The videophone connection device 18 is a device which 
connects the display device 12 and the video synthesizer 17 
with the external device connecting terminal of the videophone 
terminal 10. The videophone connection device 18 supplies a 
video signal being received by the videophone terminal 10 to 
the display device 12 , an d as well as supplies a video signal 
from the video synthesizer 17 to the videophone terminal 10. 
Thus_^_ the display device 12 defines is to be sef-ves-a-s-an 
external video display device of the videophone terminal 10 
and the target imaging camera 16 and the sign language imaging 

camera 14 defines — is to be an serve as -e x t e r n a 1 v ideo 

input imaging devices- of the videophone terminal 10. 

When such a video input/output videophone sign language 
device — #er — conversation assistance d evice f orwi-t-h — sign 
language conversation is connected to a videophone terminal 
and a sign language conversation with — sign — la ngu a g e — is 
initiated, the deaf-mute person can transmit a target video 
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a long with a sign language explanation of the target video by 
sign — language — added to the conversation partner. This 
provides the same advantages as that obtained by an unimpaired 
person's aural explanation of addcd to the target video. As 
a result, a shorter speedy conversation is 

r e quirod achieved . ma-d-e— -pes-s-i-b-l-e— Further, it is possible to 
transmit information about ef^t4ie4af^e^46"the target to the 
conversation partner e p po nen-t—p a-r-ty in a_more efficient and 
precise manner . # ashion . 

While the fixture 1 3 for fixing the display device 1 2 
in front of the eyes of a deaf-mute person uses a spectacle 
frame structure in the above-described preferred embodiment, 
the fixture 1 3 may includecomprise a hair band fixed to the 
head equipped with an arm for supporting the display device 
12, or may^-ha v e any suitable structure as long as -i-t—ea- n fi -x- 
the display device 12 can be fixed in front of the eyes of the 
deaf-mute person. 

While the target imaging camera 16 is fixed to the side 
of the fixture 13 in the above -describe d preferr e d embodiment 
example, the present invention is not limited thereto . The 
bu t th e target imaging camera 16 may be fixed to the head of 
the deaf-mute person separately from the fixture 13. 

While the sign language imaging camera 14 
includes comprisoG the waist fixture 15 fixed at the waist of 
the deaf-mute person in the above-described preferre d 
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embodiment, the sign language imaging camera 14 may use any 
type of fixing device m eans as long as it can capture p ick up 
the sign language of the deaf-mute person-a-n-d-p^ev-ird es the sam e 
effect of the inv ention. 

While the target imaging camera 16 for c ap t u r i ng pi-ek-i-n-g 
up a target for a conversation other than sign language is 
provided in the above -described preferred embodiment, an 
external video signal input terminal for inputting external 
video signal may be provided and a video signal input from the 
external video signal input terminal and a video signal from 
the sign language imaging camera 14 may be synthesized by the 
video synthesizer 17 for transmission to the conversation 
partner. With this configuration, it is possible to display 
a video from an external camera or a video from a VTR as a target 
for the conversation and discussion with the partner about e-R 
the contents of the video via w ith sign language— -irs— aiiow-ed . 

While the videophone connection device 18 connects the 
display device 1 2 and the video synthesizer 1 7 with the external 
device connecting terminal of the videophone terminal 10, via 
wires in the above -described preferred embodiment, a radio 
communications device for wirelessly transmit ting/re ceiving 
a video signal may be provided on each of the external device 
connecting terminal of the videophone terminal 10, the fixture 
13 and the video synthesizer 17. This eliminates the need for 
cables to be connected^ toc ab ling the videophone terminal 10, 
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the fixture 1 3 , and the video synthesizer 1 7 , which facilitates 
provides extreme case — e^— h andl i ng of the device . 

Where— £n— — case the videophone terminal 1 0 

includes c o mp risos- a wireless interface conforming to a 
s t a nd a r d S-t-an<feri?d- such as Bluetooth® for communicating with an 
external device, a communications device conforming to the same 
s t andar d -S-t^d-a-r-d- should be provided on each of the fixture 13 
and the video synthesizer 17. By doing so, it is possible to 
communicate a video signal without physically connecting 
anything to the videophone terminal 1 0 as long as the 
communications devices provided on the fixture 1 3 and the video 
synthesizer 17 are within the service area of the wireless 
interface of the videophone terminal 10, which further 
f a^ilitates_ adds to the case of handling. 

While a videophone terminal of a telephone type, 
especially a videophone terminal of a cellular phone type is 
used in the above -described preferred embodiment, the present 

invention is not limited thereto^ A but a videophone terminal 

of the IP type to connect to the internet may also be -I-n-terne-t 
m . a .y„...^ e ._... e .q. ua -l--l-y- used . 

While the above-described „P re Jl erre d embodiment 
describes a video — input/ outp ut — device — £-e~r — sig n — languag e 
converGation v ideophone sign language conversation assistance 
device including convcrGat ion with sign language comp rising a 
sign language imaging camera 14, a target imaging camera 16, 
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a video synthesizer 17, a display device 12, a fixture 13, and 
a videophone connection device 18, wherein char act prized in 
that the video input/output device for conversation w-±-th sign 
language convcrsat ion videophone sign language conversation 
assistance device includes both a function to synthesize a sign 
language video and a target video and supplying the resulting 
video to the videophone terminal 10 and a function to acquire 
a sign language video being received by a videophone terminal 
10 and display the sign language video on the display device 
12, a video input device for sign language conversation 
including c o nver satio n — with sign l anguag e co mprisi -ng^ a sign 
language imaging camera 14 for picking up sign language, a 
target imaging camera 16 for picking up a target other than 
sign language, a video synthesizer 17 for synthesizing a video 
from the sign language imaging camera 14 and a video from the 
target imaging camera 16, and a videophone connection device 
1 8 for supplying the synthesized video signal to the videophone 
terminal 10 allows the deaf-mute person to provide a sign 

language perform- exp 1 ana t ion by sign language whi le 

transmitting the video of a target other than sign language 
to the conversation partner, opp one n-t^a-^^yv-whire-h-p r o v i de-s— fche 
e-f-f-ee-t— e^-th-c invent -i-e^— 

Next, a sign language interpretation system will be 
described which enables allowo selection of a sign language 
interpreter satisfying the object of a conversation whenin case 
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a deaf-mute person converses with a non-deaf -mute person via 
a sign language interpreter by using a video — input/output 
device for sign language conversation videophone sign language 

converstation assistance device, conversation with sign 

■l anguag e acc o rd i ng to t h e i nv ention. 

Fig. 3 is a system block diagram of a sign language 
interpretation system according to a pref erred a-n- embodiment 
of the invention. In Fig. 3, a-numeral 100 represents a sign 
language interpretation system installed in a sign language 
interpretation center which provides a sign language 
interpretation service. The sign language interpretation 
system 100 interconnects, via a public telephone line 40, a 
videophone terminal for deaf-mute persons used by a deaf-mute 
person A (hereinafter referred to as a deaf-mute person 
terminal) 10, a videophone terminal for non-deaf -mute persons 
used by a non-deaf -mute person B (hereinafter referred to as 
a non-deaf -mute person terminal) 20, and a videophone terminal 
for sign language interpreters used by a sign language 
interpreter C (hereinafter referred to as a sign language 
interpreter terminal) 30 in order to provide a sign language 
interpretation service in a videophone conversation between 
a deaf-mute person and a non-deaf -mute person. In this 
preferred embodiment , each of the deaf-mute person terminal 
10, non-deaf -mute person terminal 20 and sign language 
interpreter terminal 30 is pref erably a telephone-type 
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videophone terminal to be connected to a public telephone line, 
and in particular^ a wireless videophone terminal of the 
cellular phone type-w&ir ch can b e ca r -r^ed-#€Hr-\^s-e— en— th e road. 

Such a videophone terminal connected to a public line 
may be an ISDN videophone terminal based on ITU-T 
recommendation H.320, the present invention is not limited 
thereto and may use a videophone terminal which usesempi^y-s- 
a unique protocol. 

When the video input/output device for conversation with 
sign language is connected to the deaf-mute person terminal 
1 0 and the deaf-mute person A wears the fixture 1 3 and the waist 
fixture 15, a sign language video received by the deaf-mute 
person terminal 10 is displayed on the display device 12 fixed 
in front of the eyes of the deaf-mute person A. The target 
imaging camera 1 6 for picking up the area in the direction of 
sight line of the deaf-mute person A and the sign language 
imaging camera 14 for picking up the sign language of the 
deaf-mute person are set and a synthesized video including a 
video of the target and explanation by sign language is 
transmitted to the o ther opponen-t party. 

The non-deaf -mute person terminal 20 is a general 
videophone terminal includingeempr is i-ng a video display 
section 20a for displaying a video received from the 
other opponent party, an imaging section 20b for picking up the 
user or target, and a headset 20c for audio input /output . 
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The sign language interpreter terminal 30 is also a 
general videophone terminal having o-f-a similar configuration 
to the non-deaf -mute person terminal 20, except that the video 
display section 30a is primarily m ain l y used to view the sign 
language of the deaf-mute person A and the video imaging section 
30b is primarilym a-i-n-ly- used to pick up the sign language 
translated into by the sign language interpreter. The headset 
30c is pr imar i ly ma-i-n-l-y used to listen to the voice of the 
non-deaf -mute person B and to input the t ranslat ion of inpu k 
t-he — vo i c e — tra ns lated — into — fr^em— the sign language of the 
deaf-mute person A. 

While input/output of voice is made using a handset on 
a typical telephone-type terminal, a headset is used instead 
in order to keep free both hands of the user who perf orms w ho s e 
maj O ir — co nc e rn — ts- sign language free - In the following 
description, a terminal uses a headset fixed on the head of 
the user including a non-deaf -mute person B. While a headset 
is not shown on the deaf-mute person terminal 10, a headset 
may be used and voice communications may also be used in 
situations where as-~~well~ r -in- ease a helper is present . t-hef e-r 

The sign language interpretation system 100 
includesc ompr irs-es- a line interface 1 20 that is connected #e-r 
trh e-<^e-af— m u t c peruse n t e r m-in-a-1— t-e-eonnee-t- 1 o a deaf-mute person 
terminal (hereinafter referred to as an I/F) 120 , a line I/F 
for the non deaf mute person terminal 140 that is connected 
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t o c o nne ct to a non-deaf -mute person terminal, and a line I/F 
160 that is connected to a for the sign language interpreter 
terminal , A — 3-6-6— t-o c onne ct fe -o— a— s-ig n la ng ua ge i nte rp re te r 

terminal . £e each i^-F arc eomected a 

multiplexer/demultiplexer 122, 142, 162 for 

multiplexing/demultiplexing a video signal is connected to 
each of the line I/F 1 20, 1 40, 1 60 , an audio signal or a data 
signal, a video CODEC (coder/decoder) 124, 144, 164 for 
compressing/expanding a video signal, and an audio CODEC 126, 
146, 1 66 for compressing/expanding an audio signal. Each line 
I/F, each multiplexer/demultiplexer, and each video CODEC or 
each audio CODEC perform call control, streaming control 
compression/expansion of a video/audio signal in accordance 
with a protocol used by each terminal. 

A video synthesizer 1 28 is connected toTo the video input 
of the video CODEC 124 for the deaf mute person terminal 124 
is connected a vide o synthesizer 1 28 -for synthesizing the video 
output of the video CODEC 144 for the non-deaf -mute person 
terminal 1 44, the video output of the video CODEC 1 64 for the 
sign language interpreter terminal 164 and the output of the 
telop memory _ f o r t h-e— de-af—^RU-^e— pe-rs-en— ke-rmirBra-l— -1 3 2 for the 
deaf-mute person terminal 4^5 - is connected to the video input 
of the video CODEC 124 for the deaf-mute person terminal . 

An audio synthesizer 13 0 is connected toTo the audio 
input of the audio CODEC for the deaf mute person terminal 126 
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drs — connected an audio synthesizer — 130 for synthesizing the 
audio output of the audio CODEC 1 46 for the non-deaf -mute person 
terminal 146 and the audio output of the audio CODEC _f or the 
sign language in terpreter t er m inal 166 for the sign language 
interpreter terminal 166 is connected to the audio input of 
the audio CODEC 126 for the deaf-mute person terminal . 

While audio input/ output is not generally providedm -ade 
on a deaf-mute person terminal -"4^^ 

GGB-E-C 1 2 6-o^--thre~-aud-i-o syn t hesizer 1 30 f or th e d eaf mute person: 
-te-rffl-irn-a-l— -ma-y be o m itted-, a voice communications function is 
preferably provided in situations in which for a cas e where 
the environment sound of a deaf-mute person terminal is to be 
transmitted to a non-deaf -mute person terminal or a case where 
a helper assists the deaf-mute person. 

A -T-e — fehe — v-ide-e — ±-npti-t — e-f- — t-h-e — video — CODEC — £e-r — the 
Hen- de-a-f— mut-e — pe-r-s-en — ter-m-i-na-1 — 1-44 — i-^s — een^iee-ted — a — vi de o 
synthesizer 148 is connected to the video input of the video 

CODEC fe-he non deaf mute person terminal 1-44 for 

synthesizing the video output of the video CODEC 1 24 for the 
deaf-mute person terminal 1 2 4 , the video output of the video 
CODEC 1 46 for the sign language interpreter terminal 164 and 
the output of the telop memory_f L e-r— t&e— non— dea#— murt-e-pei^s-ef* 
te^ni-Fta-l — 152 for jthe non-deaf -mute person terminal is 
connected to the video input of the video CODEC for the 
non-deaf-mute person terminal . 
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An audio synthesizer 150 is connected toTo the audio 
input of the audio CODEC for the non deaf mute person terminal 
146 is co -rmeetred an a u d4^--syn-1^hes4-g- er 1 5 0-f or synthesizing the 
audio output of the audio CODEC 1 26 for the deaf-mute person 
terminal 1 26 and the audio output of the audio CODEC 166 for 
the sign language interpreter terminal 4-6^- is connected to the 
audio input of the audio CODEC 146 for the non-deaf -mute person 
terminal , 

While video display of a sign language interpreter may 
be omitted on a non-deaf -mute person terminal, understanding 
of the voice interpreted by the sign language interpreter is 
f acilitated m ade — eas y by displaying the video of the sign 
language interpreter, suchs -e- that a function is preferably 
provided to synthesize the video of a sign language 
interpreter . 

A video synthesizer 1 68 is connected toT - o the video input 
of the video CODEC for th e sign language interpreter terminal 
1 64 ±-s~~ee nnee ^ed--a--v-id ee-sy-n-t-hes~i-z-e-r— 1-6^8- for synthesizing the 
video output of the video CODEC 1 24 for the deaf-mute person 
terminal 124 , the video output of the video CODEC 144 for the 
non-deaf -mute person terminal 1 44 and the output of the telop 
memory 1 7 2 for the sign language interpreter terminal 4-7-2- is 
connected to the video input of the video CODEC 1 64 for the 
sign language interpreter terminal . 

An audio synthesizer 17 0 is connected — toTo the audio 
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input of the audio CODEC for the sign language interpreter 
terminal — 1-6-6 — i-s — connected — an — au di o — synthesizer — — for 
synthesizing the audio output of the audio CODEC 1 26 for the 
deaf-mute person terminal 1 26 and the audio output of the audio 
CODEC 146 for the non-deaf -mute person terminal — is 
connected to the audio input of the audio CODEC 166 for the 
sign language interpreter terminal . 

While video display of a non-deaf -mute person may be 
omitted on a sign language interpreter terminal, understanding 
of the voice in interpreting the voice of a non-deaf -mute person 
is f acilitatedmade — e-a-s-y by displaying the video of the 
non-deaf -mute person, suchse that a function is preferably 
provided to synthesize the video of a non-deaf -mute person. 

The sign language interpretation system 100 is equipped 
with a sign language interpreter registration table 182, in 
which -w-he-r-e- the terminal number of a terminal for sign language 
interpreters used by a sign language interpreter is registered 
and includes a controller 180 connected to each of the line 
I/Fs 1 20, 1 40, 1 60 , multiplexers/demultiplexer s 1 22 , 144, 1 62, 
video synthesizers 1 28, 1 48, 1 68, audio synthesizers 1 30, 1 50, 
170, and telop memories 132, 152, 172. The controller 180 
connectsprH3 vi d es a f un ction tre— ee-n-nee-^ a calling terminal, a 
sign language interpreter terminal and a called terminal via by 
way of a function to accept a call from a terminal used by a 
deaf-mute person or a terminal used by a non-deaf -mute person, 
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a function to prompt a calling terminal to enter the called 
terminal number, a function to extract the terminal number of 
a sign language interpreter from the sign language interpreter 
registration table 182, a function to call the extracted 
terminal number, and a function to call the terminal number 
of the called terminal, and also provides a function to switch 
between video/audio synthesis methods used by video/audio 
synthesizers and a function to generate a telop and transmit 
the telop to a telop memory. 

Figs. 4 (a) -4 (c) show sh o w s — an example of a video 
displayed on the screen of each terminal during a videophone 
conversation viab y way of the sign language interpretation 
system according to a pr e ferred embodiment of the present 
invention. Fig. 4(a) shows the screen of a deaf-mute person 
terminal. A video synthesizer 128 displays on the screen a 
video obtained by synthesizing a video of a non-deaf -mute 
person terminal and a video of a sign language interpreter 
terminal. While the video of the non-deaf -mute person is 
displayed as a main window and the video of the sign language 
interpreter is displayed as a sub window in a 
Picture-in-Picture fashion, a Picture-in-Picture display in 
which is a - ls o p o s-s-irbl-e— a-s g umi ng the video of the sign language 
interpreter is — as a the main window and the video of the 

non-deaf -mute person as a sub window^ _i_s also possible . 

Alternatively -: Or 1 , these videos may be displayed so as to have 
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an i^-equal size. When the video of a sign language interpreter 
is displayed in a larger size, the sign language interpreted 
by the sign language interpreter is easier to view and 
understand. A command from a terminal is preferably used to 
change the position of a sub window in the Picture-in-Picture 
display such- s-o- that the sub window will not obstruct the view 
ofma-s-k- important information in the main window. 

Fig. 4(b) shows the screen of a non-deaf -mute person 
terminal. The video synthesizer 148 displays on the screen 
a video obtained by synthesizing a video of a deaf-mute person 
terminal and a video of a sign language interpreter terminal. 
While the video of the deaf-mute person terminal is a 
Picture-in-Picture representation including the target video 
captured picked up by the target imaging camera 16, the sign 
language video captured pirek-ed-^p- by the sign language imaging 
camera 14 arranged on the lower left of the target video, and 
the video of the sign language interpreter arranged on the lower 
right of the target video. The video of the sign language 
interpreter may be omitted. By displaying the video of the 
sign language interpreter in a Picture-in-Picture fashion, the 
non-deaf -mute person can observeeteek- the expression of the 
sign language interpreter on the screen, which facilitates 
unders tanding of ma-k-e-s — i-t — easi er — to — unde^s-t-and- the voice 
translated into sign language by the sign language interpreter. 

Fig. 4 (c) shows the screen of a sign language interpreter 
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terminal. The video synthesizer 168 displays on the screen 
a video obtained by synthesizing a video of a deaf-mute person 
terminal and a video of a non-deaf -mute person terminal. The 
In this case al so , th e video of the deaf-mute person terminal 
is a Picture-in-Picture representation including the target 
video capture dp -i-ek-ed-^u-p by the target imaging camera 16, the 
sign language video capture dpireked— u-p by the sign language 
imaging camera 14 arranged on the lower left of the target video, 
and the video of the non-deaf -mute person arranged on the lower 
right of the target video. The video of the non-deaf -mute 
person may be omitted. By displaying the video of the 
non-deaf -mute person in a Picture-in-Picture fashion, the sign 
language interpreter can observe c hcck the expression of the 
non-deaf -mute person on the screen, which facilitates 
understanding of m ak e s it e asier to-u-nde^s-tendr the voice of the 
non-deaf -mute person as a target for sign language 
interpretation . 

T o fe-~-0f.4ef-""40--- support a situation in which ea-se where 

the environmental sound of a deaf-mute person terminal is to 

be transmitted or a situation in which case where- a helper 

assists the deaf-mute person, a voice obtained by synthesizing 
the voice from the non-deaf -mute person terminal and the voice 
from the sign language interpreter terminal by using the audio 
synthesizer 130 is output to the deaf-mute person terminal, 
a voice obtained by synthesizing the voice from the deaf-mute 
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person terminal and the voice from the sign language 
interpreter terminal by using the audio synthesizer 150 is 
output to the non-deaf -mute person terminal, and a voice 
obtained by synthesizing the voice from the non-deaf -mute 
person terminal and the voice from the deaf-mute person 
terminal by using the audio synthesizer 170 is output to the 
sign language interpreter terminal. 

When - -I-n~ -ease it is not necessary to transmit the 

environmental sound of the deaf-mute person terminal or a 
helper is not present, the audio synthesizers 1 30, 1 50 and 170 
may be omitted and the output of the audio CODEC 146 for the 
non-deaf -mute person terminal 1 4 6 may be connected to the input 
of the audio CODEC 166 for the sign language interpreter 
terminal 1 66 and the output of the audio CODEC 1 66 for the sign 
language interpreter terminal 1 66 may be connected to the input 
of the audio CODEC 1 4 6 for the non-deaf-mute person terminal 

Operation of the video synthesizers 128, 148, 168 and 
audio synthesizers 1 30, 1 50, 1 70 is controlled by the 
controller 180. The user may change the video output method 
or audio output method by pressing a predetermined number 
button on e# — a dial pad of each terminal. This is 
ini tiated-imp-lefflef^ted- when a push on the number button on the 
dial pad of each terminal is detected as a data signal or a 
tone signal by the multiplexer/demu ltiplexer 1 22, 1 44, 1 62 and 
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detection of the push on the button is signaled to the 



controller . 

With this configuration, flexibility in the usage of the 
system on each terminal is ensured. For example, only 
necessary videos or audios are selected and displayed/output 
in accordance with the object or it is possible to replace a 
main window with a sub window, or change the position of the 
sub window. 

A To t h e input of the audio syn t hesiz e rs 128, — — 1-6-8- 



per son 132 , a telop memory 1 52 for the non-deaf -mute person 
4-53-, and a telop memory 172 for the sign language interpreter 
4-74*- are respectively connected to the input of the audio 
synthesizers 128, 148, 1 68 . Contents of each telop memory 132, 
1 52, 172 are ean be set byf-pom- the controller 180. 

With this configuration, by setting a message to be 
displayed on each terminal to the telop memories 1 32, 1 52, 1 72 
and issuing an instruction to select a signal of the telop 
memories 1 32, 1 52, 1 72 to the audio synthesizers 1 28, 1 48, 1 68 
in the setup of a videophone conversation via sign language 
interpretation, it is possible to transmit necessary messages 
to respective terminals to establish a three-way call. 

In situations in whichease there is a difficult term 
which is dif f icult h ard to explain must be explained using sign 
language or a word term that which is dif f icult hard to pronounce 




telop memory 1 3 2 for the deaf-mute 
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in a videophone conversation, it is possible to register in 
advance the tcrm these terms may be registered term in the term 
registration table 184 of the controller 180 corresponding to 
a in association—with the number one-f the dial pad on each 
terminal. By doing so, it is possible to detect a push on the 
dial pad on each terminal during a videophone conversation, 
extract the term corresponding to the number of the dial pad 
pressed from the term registration table, generate a text telop, 
and set the text telop to each telop memory, thereby displaying 
the term on each terminal. 

With this configuration, a term which is d ifficult h ar d 
to explain using sign language or a word — term which is 
dif f icult hard to pronounce is transmitted via by way of a text 
telop to the conversation partner oppon e n t — part y , thus, 
providing a quicker and more to-the-point videophone 
conversation. 

Next , a processing flow of the controller 180 for setting 
a videophone conversation via sign language interpretation is 
explained. 

Prior to processing, information to select a sign 
language interpreter and the terminal number of a terminal used 
by each sign language interpreter are registered in the sign 
language interpreter registration table 182 of the controller 
180 from an appropriate terminal (not shown). Fig. 6 shows 
an example of ^registration item to be registered in the sign 
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language interpreter registration table 182. The information 
to select a sign language interpreter refers to information 
used by the user to select a desired sign language interpreter, 
which includes a-sex, an— age, a— habitat ion, a— specialty, and 
the level of sign language interpretation skill. Habit at ion — 
-T-he-h a b i t a-t-ien assumes a situation in whiche a-s-e-w-he^e- the user 
wants des ir e s a person who has geographic knowledge of on a 
specific area and, in this example, a ZIP code is used to specify 
an area. Speci a 1 ty Th e spe cialt y assumes a situationeas-e-whe-re- 
in whichea-s c the con vers a ti on pert ai ns to a spe cific f-i-e-l-d-r 
the user wants d e sires a person who has expert knowledge in a 
particularon the field or is familiar with the topics in that t-he 
field. In this example, the fields in which a sign language 
interpreter is familiar with are classified into several 
categories to be registered, such as politics, law, business, 
education, science and technology, medical care, language, 
sports, and hobby. The specialties are diverse, such- s-e that 

they may be registered hierarchically and searched through at 

a level desired by the user when selected. 

In addition, qualifications of each sign language 
interpreter may be registered in advance for the user to select 
a qualified person as a sign language interpreter. 

The terminal number to be registered is the telephone 
number of the terminal, because in this example a videophone 
terminal to connect to a public telephone line is 
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provided. assumed. 

In the sign language interpreter registration table 1 82^ 
4_s — pi^e v ided — an availability flag is provided to indicate 
whether sign language interpretation can be accepted. A 
registered sign language interpreter can call the sign language 
interpretation center from his/her terminal and enter a command 
by using a dial pad to set/reset the availability flag. Thus, 
a sign language interpreter registered in the sign language 
interpreter registration table can set the availability flag 
only when he/she is available for sign language interpretation, 
thereby eliminating useless calling and permit ting al lowin g 
the user to select an available sign language interpreter 
without delay. 

Fig. 5 shows a p roc e sspro cc s s i ng flowchart of the 
controller 180. The sign language interpretation system 100 
allows a deaf-mute person terminal or non-deaf -mute person 
terminal to propose a sign language interpretation service. 
From the deaf-mute person terminal, the user places a call to 
a telephone number on the line I/F for the deaf-mute person 
terminal. From the non-deaf -mute person terminal, the user 
places a call to a telephone number on the line I/F for the 
non-deaf -mute person terminal. This calls the sign language 
interpreter terminal and the part ner's eppe^e-n-t- terminal and 
establishes a videophone connection via sign language 
interpretation . 
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As shown in Fig. 5, it is first detected that the line 
I/F 1 20 for the deaf-mute person terminal 1 20 or line I/F 140 
for the non-deaf -mute person terminal 1 4 0 that is called is 
detected first (SIOO) . Next, the calling terminal displays a 
screen to prompt input of the terminal number of the called 
party shown in Fig. 7 (S1 02 ). ^The terminal number of the called 
party input by the caller is acquired (S104). The calling 
terminal displays a screen to prompt input of the selection 
conditions for a sign language interpreter shown in Fig. 8 
(S1 06) . The sign language interpreter selection conditions 
input by the caller are acquired (S108). The sign language 
interpreter selection conditions input by the caller are sex, 
age bracket, area, specialty and sign language level. A 
corresponding sign language interpreter is selected based on 
the sex, age, habitation, specialty, and sign language level 
registered in the sign language interpreter registration table 
182. The area is specified by using a ZIP code and a sign 
language interpreter is selected starting with the habitation 
closest to the specified area. For any selections, i fin-ease 
it is not necessary to specify a condition, N/A may be selected. 

Next, a sign language interpreter with availability flag 
set is selected from among the sign language interpreters 
satisfying the selection conditions acquired referring to the 
sign language interpreter registration table 1 82 . The calling 
terminal displays a list of sign language interpreter 
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candidates shown in Fig. 9 to prompt input of the selection 
number of a desired sign language interpreter (SI 1 0 ) . The 
selection number of the sign language interpreter input by the 
caller is acquired ( SI 1 2 ) and the terminal number of the 
selected sign language interpreter is extracted from the sign 
language interpreter registration table 182 and the terminal 
is called (S114) . When the sign language interpreter terminal 
has accepted the call (S116), the called terminal number is 
extracted and called (S118). When the called terminal has 
accepted the call (S120), a videophone conversation via sign 
language interpretation starts (S122). 

When^ -n — ea-s-e the sign language interpreter terminal 
selected in S1 66 does not accept the call, whether a next 
candidate is available is determined (S124). When ln case a 
next candidate is available, execution returns to S114 and the 
procedure is repeated. Otherwise the calling terminal is 
notified as such and the call is released (S126). 

When — -In— ease the called terminal does not accept the 

call in S120, the calling terminal and the selected sign 
language interpreter terminal are notified as such and the call 
is released (S1 28) . 

While wheni-n case the selected sign language interpreter 
terminal does not accept the call, the caller is notified as- 
such and the call is released in the above-described preferred 
embodiment, a sign language interpretation reservation table 
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to register a calling terminal number and a called terminal 
number may be provided and the caller and the called party may 
be notified on a later response from the selected sign language 
interpreter to set a videophone conversation. 

While the sign language interpretation system 100 
inc ludes G-empr~i~se-s~ a line I/F, a multiplexer /demultiplexer , a 
video CODEC, an audio CODEC, a video synthesizer, an audio 
synthesizer and a controller in the above -described preferred 
embodiment, these components need not be prov id ed i^ftpjre-meffrt-ed 
by individual hardware (H/W)^ but rather, the function of each 
component may be provided via implemented by i mp lemef*-t-ed-b a3cd - 
ob— software running on a computer. 

While the sign language interpreter terminal 30, same 
as — the — deaf mute person — t e rminal — 4-0 — a**d — t-he — n o n de af mute 
pe-r-se-n — t-e rmin a-1 — 2-£b — is located outside the sign language 
interpretation center and called from the sign language 
interpretation center over a public telephone line to provide 
a sign language interpretation service in the above -prefer red 
embodiment, the present invention is not limited thereto, and 
a portion -bu-t— pa-r-t- or all of the sign language interpreters 
may be provided in the sign language interpretation center to 
provide a sign language interpretation service from the sign 
language interpretation center. 

In the above -described preferred embodiment, a sign 
language interpreter may e-a-n join a sign language 
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interpretation service anywhere he/she may be, as long as 
he/she has a terminal which can be connected to a public 
telephone line. Thus^_ the sign language interpreter can 
provide a sign language interpretation service by using the 
availability flag to make efficient use of free time. By doing 
so, it is possible to stably operate a sign language 
interpretation service accompanied by a problem of difficult 
reservation of a sign language interpreter. In particular, 
the number of volunteer sign language interpreters is 
increasing nowadays. A volunteer who is available only 
irregularly can provide a sign language interpretation service 
by taking advantage of their a— limited free time. 

While a video signal of the home terminal is not input 
to the video synthesizers 1 28, 148, 168 in the above- described 
preferred embodiment, a function may be provided to input the 
video signal of the home terminal for later synthesis and 
display to check the video on the terminal. 

While the video synthesizers 1 28, 1 48, 1 68 and the audio 
synthesizers 130, 150 170 are used to synthesize videos and 
audios for each terminal in the above -described preferred 
embodiment, the present invention is not limited thereto. 
Video— bu-^vi-deos* and audi o audiop from all terminals may be 
synthesized at the same ti meenee and the resulting video or 
audio may be transmitted to each terminal. 

While a function is provided such that w hereb y the telop 
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memories 132, 152, 172 are provided and telops are added to 
the video synthesizers 1 28, 148, 1 68 in order to display a text 
telop on each terminal in the above -de scribed preferred 
embodiment, a function may be provided whereby a telop memory 
to store audio information and telops are added to the audio 
synthesizers 130, 150, 170 in order to output an audio message 
on each terminal. This makes it possible to set a videophone 
conversation via sign language interpretation even in case the 
non-deaf-mute person is a visually impaired person. 

Fig. 1 0 is a system block diagram of a sign language 
interpretation system according to another p r efe r r ed 
embodiment of the prese n t invention. This preferred 
embodiment shows a system configuration example which 
assumes assuming that each terminal used by a deaf-mute person, 
a non-deaf -mute person and a sign language interpreter is an 
IP (Internet Protocol) type videophone terminal to connect to 
the internet I nter -ne£— equipped with a web browser. 

In Fig. 10, a numeral 200 represents a sign language 
interpretation system installed in a sign language 
interpretation center to provide a sign language 
interpretation service. The sign language interpretation 
system 200 connects a deaf-mute person terminal 50 used by a 
deaf-mute person, a non-deaf -mute person terminal 60 used by 
a non-deaf -mute person, and the selected sign language 
interpreter terminals used by a sign language interpreter 231 , 
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232, ... via the Internet 70^_ in order to provide a videophone 
conversation service via sign language interpretation between 
the deaf-mute person and the n on-deaf -mute person. 

While each of the deaf-mute person terminal 50, the 
non-deaf -mute person terminal 60 and the sign language 
interpreter terminals 231, 232,... includese-ae-h — eompr ise-s- a 
general-purpose processing device (a) such as a personal 
computer having a video input I/F function, an audio 
input/output I/F function and a network connection function, 
a keyboard (b) and a mouse (c) for input of information as well 
as a display (d) for displaying a web page screen presented 
by a web server 210 and a videophone screen supplied by a 
communications server 220, a televi sion camera (e) for imaging 
the sign language of a sign language interpreter, and a headset 
(f) for performing audio input /output for the sign language 
interpreter, the processing device include s -ha-s- IP videophone 
software and a web browser installed in this example, a 
dedicated videophone terminal may be used instead. 

The videophone terminal connected to the 
internet -I-n-te-ra e -t- may be an IP videophone terminal based on 
ITU-T recommendation H.323, the present^ invention is not 
limited thereto L and— but- may use a videophone terminal which 
°P3 1 tP i ^3?^^? co ^^ n 9^ toe^n-plre-ys- a unique protocol. 

The internet Internet may be e#-a wireless LAN type . The 
videophone terminal may be a cellular phone or a portable 
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terminal equipped with a videophone function and also including 
a web access function. 

The sign language interpretation system 200 
includes co mpr i s c s : a communications server 220 including a 
connection table 222 for setting the terminal addresses of a 
deaf-mute person terminal, a non-deaf -mute person terminal and 
a sign language interpreter terminal as well as a function to 
interconnect the terminals registered in the connection table 
222 and synthesize a video and an audio received from each 
terminal and transmit the synthesized video and audio to each 
terminal ,7- a web server 210 including a sign language 
interpreter registration table 212 for registering the 
selection information, terminal address and availability flag 
of a sign language interpreter as mentioned earlier, as well 
as a function to select a desired sign language interpreter 
based on an access from a calling terminal by using a web browser 
and set the terminal address of each of the calling terminal, 
called terminal and sign language interpreter terminal in the 
connection table 222 of the communications server 
220, ser-ve-r-2-2-Q-t- a router 250 for connecting the web server 210 
and the communications server 2 20 to the internet , Int e-p-n-e^-f- 
and a plurality of sign language interpreter terminals 231 , 
232,..., 23N connected to the communications server 220 via a 
network . 

Fig. 11 shows an example of a connection table 222. As 
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shown in Fig. 11, the terminal address of a deaf-mute person 
terminal, the terminal address of a non-deaf -mute person 
terminal and the terminal address of a sign language 
interpreter terminal are registered as a set in the connection 
table 222. This provides a single sign language 

interpretation service. The connection table 222 is designed 
to register a plurality of such terminal address set depending 
on the throughput of the communications server 220, thereby 
simultaneously providing a plurality of sign language 
interpretation service s . 

While the terminal address registered in the connection 
table 222 is an address on the interne t Internet and is generally 
an IP address, the present invention is not limited thereto^ 
and, but for example^ a name given by a directory server may 
be used. 

The communications server 220 performs packet 
communications using a predetermined protocol with the 
deaf-mute person terminal, non-deaf -mute person terminal and 
sign language interpreter terminal set to the connection table 
222 and provides, by way-ef-sof tware processing, the functions 
similar to those provided by a multiplexer /demultiplexer 122, 
142, 1 62, a video CODEC 1 24, 144, 1 64, an audio CODEC 1 26 , 1 46, 
166, a video synthesizer 128, 148, 168, an audio synthesizer 
1 30, 1 50, 1 70 in the above -described sign language 
interpretation system 100. 
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With this configuration, similar to s-ame — as— the sign 
language interpretation system 100, prescribed videos and 
audios are communicated between a deaf-mute person terminal, 
a non-deaf -mute person terminal and a sign language interpreter 
terminal, and a videophone conversation via sign language 
interpretation is established between the deaf-mute person and 
the non-deaf-mute person. 

While the sign language interpretation system 1 00 uses 
the controller 180 and the telop memories 132, 152, 172 to 
extract a term registered in the term registration table 184 
during a videophone conversation based on an instruction from 
a terminal and displays the term as a telop on the terminal, 
the same function may also be provided via by way of software 
processing by the communications server 220 in this pr eferr ed 
embodiment als o, A term specified by each terminal may be 
displayed as a popup message on the other terminal by way of 
the web server 210. Or, a telop memory may be provided in the 
communications server 220 suchse that a term specified by each 
terminal will be written into the telop memory via the web 
server 210 and displayed as a text telop on each terminal. 

While the sign language interpretation system 1 00 uses 
the controller 1 80 to interconnect a deaf-mute person terminal, 
a non-deaf -mute person terminal and a sign language interpreter 
terminal, the connection procedure is made by the web server 
210 in this preferred embodiment because each terminal has a 
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web access function. 

Fig. 1 2 is a processing flowchart of a connection 
procedure by the web server 210. The sign language 
interpretation system 200 also enables a Hows a deaf-mute 
person terminal or non-deaf -mute person terminal to 
request prepes e- a sign language interpretation service. A 
deaf-mute person or a non-deaf -mute person wishing to 
request p-ropes-e- a sign language interpretation service 
accesses the web server 21 0 in the sign language interpretation 
center by-using a web browser to log in from each own terminal, 
which starts the acceptance of the sign language interpretation 
service . 

As shown in Fig. 12, the web server 210 first acquires 
the terminal address of a caller (S200) and sets the terminal 
address to the connection table 222 (S202) . Next, the web 
server delivers a screen to prompt input of the called terminal 
address similar to that shown in Fig. 7 to the calling terminal 
(S204) . The called terminal address input by the caller is 
acquired (S206) . The web server delivers a screen to prompt 
input of the selection conditions for a sign language 
interpreter similar to that shown in Fig. 8 to the calling 
terminal (S208) . The sign language interpreter selection 
conditions input by the caller are acquired (S210). 

Next, a sign language interpreter with an availability 
flag set is selected from among the sign language interpreters 
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satisfying the selection conditions acquired from the sign 
language interpreter registration table 212. The web server 
210 delivers a list of sign language interpreter candidates 
similar to that shown in Fig. 9 to the calling terminal to prompt 
input of the selection number of a desired sign language 
interpreter (S212) . The selection number of the sign language 
interpreter input by the caller is acquired and the terminal 
address of the selected sign language interpreter is acquired 
from the sign language interpreter registration table 212 
(S214). Based on the acquired terminal address of the sign 
language interpreter, the web server 210 delivers a calling 
screen to the sign language interpreter terminal (S216) . Ifl-n 
case the call is accepted by the sign language interpreter 
(S218) , the terminal address of the sign language interpreter 
is set to the connect ion Gonnec t i o n table 222 (S220). Next, 
the web server 210 delvers a calling screen to the called 
terminal based on the acquired called terminal address (S222) . 
I f -I-n--ea-se the call is accepted by the called terminal (S224) , 
the called terminal address is set to the connection table 222 
(S226) . Then, a videophone conversation via sign language 
interpretation beginss tarts (S228) . 

If-£ n cas e the sign language interpreter terminal does 
not accept the call in S218, whether a next candidate is 
available is determined (S230) . If ln case a next candidate 
is available, the web server delivers a message to prompt the 
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caller to select another candidate (S232) to the calling 
terminal, a nd the thcn execution returns to S214. If ln case 
another candidate is not found, the calling terminal is 
notified as such (S234) and the call is released. 

If ln case the called terminal does not accept the call 
in S224, the calling terminal and the selected sign language 
interpreter terminal are notif ied as-sueh- (S236) and the call 
is released. 

When — Wfeiie ±n cas e the selected sign language 

interpreter terminal does not accept the call, the caller is 
notified as such and the call is released in the above -described 

preferred embo dim ents em bodiment , a s i gn 1 angu age 

interpretation reservation table to register a calling 
terminal address and a called terminal address may be provided 
and the caller and the called party may be notified ofef* a later 
response from the selected sign language interpreter to set 
a videophone conversation. 

While the sign language interpreter terminal is located 
in the sign language interpretation system 200 of the sign 
language interpretation center in the above -described 
preferred embodiment s-embodime-n^, the present invention is not 
limited thereto, and -^u-^-some or all of the sign language 
interpreter terminals may be provided outside the sign language 
interpretation center and connected via the Internet. 

In the above -de scribed preferr ed e m bod iments embodiment , 
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the configuration of the sign language interpretation system 
has been described for a situat ion in which casQ — w here a 
videophone terminal used by a deaf-mute person, a non-deaf -mute 
person or a sign language interpreter is a telephone-type 
videophone terminal connected to a public telephone line^ and 
a situation in which the ea-s-e---wher^e~~t"he~~v ideophone terminal 
is an IP-type videophone terminal connected to the Internet, 
the telephone-type videophone terminal and the IP-type 
videophone terminal can communicate with each other by 
arranging a gateway to perform protocol conversion 
therebetween. A sign language interpretation system 
conforming to one protocol may be provided via the gateway to 
support a videophone terminal conforming to the other protocol . 

In this mannerway-, the sign language interpretation 
system enablesa ^rl-ews- the user to enjoy or provide a sign 
language interpretation service anywhere he/she may be, as long 
as he/she has a terminal which can be connected to a public 
telephone line or the internet. - i-H-t-er-ne-t-.-- A sign language 
interpreter does not always have to visit a sign language 
interpretation center but can present a sign language 
interpretation from his/her home or a facility or site where 
a videophone terminal is located, or provide a sign language 
interpretation service by using a cellular phone or a portable 
terminal equipped with a videophone function. 

A person with the ability of sign language interpretation 
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may wish to register in the sign language interpreter 
registration table in the sign language interpretation center 
in order to provide a sign language interpretation service 
whenever an yt -if n e wh en-it is convenient to him/her. From the 
viewpoint of the operation of the sign language interpretation 
center, it is not necessary to summon sign language 
interpreters to the center. This provides a-l-l-ows- efficient 
operation of the sign language interpretation center both in 
terms of time and costs . In particular , the number of volunteer 
sign language interpreters is increasing nowadays. The sign 
language interpretation service can be provided from a sign 
language interpreter's home, which facilitates reservation of 
a sign language interpreter. 

^n dustri a-l— Ap p lic abi l i ty 

As mentioned above, according to preferred embodiments 
of the present invention, a deaf-mute person can include an add 
explanation with sign language while transmitting a target 
video other than sign language. It is thus possible to 
precisely explain the target to thereby speed sp e ed-in g up a 
conversation. 

Whi le t he present inv enti on has been described w ith 
respect to preferred embodiments, it wi ll be apparent to those 
skilled in the art that Jbhe disclosed inv ent ion may be modified 
in numerous w ays and may as sume man y embodim ent s other than those 
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specifically set out and described above. Accordingly, it is 
intended by the appended claims to cover all modifications of 
the invention which fall within the true spirit and scope of 
the invention. 
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WHAT IS CLAIMED IS : 



1_. A videophone sign language conve rsation 

assistance device used by a deaf-mute person to have a sign 
language conversation using of a videophone comprising: 

hand imaging means including waist fixing means to be 

fixed at the waist of a deaf-mute person to capture images of 
the hands of said deaf-mute person to acquire a sign language 
video; 

si ght _1 ine di recti on im agi ng me a ns f ix e d to the h ead of 

sa_id deaf-mute person and arranged to capture images of an area 
in a directi on o f__ the sig h t li ne of said deaf-mut e person; 

video sig nal s ynth esis means for synthesizing a vi de o 

signal acquired captured b y s a id hand imaging means and a video 
s ignal picked up captured by sa id sight li ne di rection i maging 
means; and 

a videophone connection means for transmitting a video 

signal obtained through synthesis by said video signal 
synthesis means to a videophone terminal; wherein 

the deaf -mu te pers on can include a n exp l anation by sig n 
language while transmitting a video in the sight line 
direction. 
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CLAI M S 

4-: A video input device for conversation with sign language 

eempri-s-i-ng-: — s-ign lan guag e i ma-girng—fne ans f op— p-i-e-k-arng— up sign 
la nguag e ; vi drc o a c qui sitio n mea n s for a c quiring a vid e o o t h e r 
th-a-n sign languag e; v-i-deo-s-ynthcGi-s-me-a-nG for synth e sizin g-a 

fnea^«^i^d^^^ee™e^e-r^ 

-vi-dee™aequ~iS"i~t^-on-™means^ — a-nd^v4^ee— means — f-er 

tr an s m itti ng the vide o s y nt h esize d by s aid v i d e o synthes-i-s- 
means; — eh-a-r-a-e-t-e-zr-i-^ed — in — t hat — a — deaf— mu-te — pe-r-s-en — ea-n — add: 
exp lanation — w-h-ti e — transmi tti ng — a — video ot h e r — t-han — s-ig-n 
langua g e . — 

2. The__ videophone _jsign language conve rsa tion assistance 

device according to claim 1 , wherein said video signal 
synthesis means includes a function to synthesize a video 
signal acquired captured by said sight line direction imaging 
means as a main window and a video signal acquired captured 
b y said hand imaging means as a sub window in a 
Pic ture- in-Picture arrangement and a feature — function to 
change jthe setting _of the posjltion of said sub window^e-^virdeo 
in p u t de-vire e for conv er sa tion- with sign l a n guage- according t o 
ej.-e-i-m 1 , c h^r-ae-te-r-i-z-ed-~i-R t hat s-a-idHS-i-g-n-lrang-uraqe--: imag-i-ng-fftean-s- 
comprises waist fixing means to be fixed at the waist of said 
deaf mute p e rson . 
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3 . The videophone sign language conv ersation 



assistance device according to claim 1 , wherein said videophone 
sign language conversation assistance device includes display 
means fixed to the head of said deaf-mute person for displaying 
a video received by said videophone terminal in front of the 
eyes of the said deaf-mute person and simultaneously enabling 
the deaf-mute person to view the outer world including a target 
for sign language conversati on ; an d The video inpu t d e v i c e fo r 
eo nv er sat i o n— - with sign langua ge ae eoirefe^g — £ o clai m — 1 — or — 2~r 
c ha ra cterized--ar n that said v id eo transmission m e an s comp rise s 
v i d eop hon e connection moan s to b e—eora*ect e d to a v i d e o phone 
terminal . 

said videopho ne c onne c t ion means i nclude s a fun ct i on to 
receive a video signal from said videophone terminal and supply 
the video signal to said display means. 

4_. The videophone sign language conversation 

assistance device accor di ng to claim 3, wherein sai d sight line 
direction imaging means and said display means are molded into 
a f rame whic h can be fixed to the ears__and nose of sa^id deaf-mute 
person. 



4- ! fhe — v ideo — input — de vic e — fe*= — eof wersation — w it h — si gn 
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lan guag e — according to — cl aim — 3-? — characterized — in — that — said 
videop hone connection means comprises radio communicat ions 
means f or^-perf o r roing rad-ie-eommu-mre-a-t-ion g with s a^d-v- Me o p -hone 
ter m inal . — 

5 . The videophone sign language conversation assistance 
device according to claim 1 , wherein said videophone connection 
means includes radio communications means for performing radio 
communications^ _with said ^videophone terminal. A — v-idee 
input/outpu t — devi-ee — fe-r — eo-iw ers a tio n — wi-t-h — s-i-g-n — ta-ng-uage- 
c ompri s ing t h e v -i d e o input device for convcr s a ti o n - wi th sign 
language accor di n g to — claim 3 — or 4 , 

6. A videophone sign language interpretation system 
connecting the videophone sign language conversation 
assistance device according to claim 1 with the videophone 
terminal of a deaf-mute person and interconnecting the 
videophone terminal of said deaf-mute person, the videophone 
terminal of a non-deaf -m ute person and the videophone terminal 
of a sign language interpreter in order to provide sign language 
interpretation by a sign language_interpreter in a videophone 
conversation between a deaf-mute person and a non-deaf -mute 
per so n, whe rein 

said videophone sign language _ interpretation — said 
v ideo ph o n e — co nn ectio n — m e ans — incl u din g — si gn — l anguag e — vid e o 
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r e cei ving mean s — — receiving — a — sign — lan guag e — video — b eing 
r e ceived by said videophone t e rminal, — characterized in that 

said v -i-d-ee- i npu t/ output — dev-i-ee — §©-r — c on v er sat i on with 

sign language includes display me an s — for disp l ay i ng a sign 

means------aRd^ 

— — Th e vid ee-irnp-u^e utput devic e for ©© nversat ion with sign 
langua ge a ccor di ng — to — e-la-im — h- f — c h ar ae^er-ir-a-ed — i-n — tha-t — sa-i-d 
display—m eans comprises a convex lens which can s ubs tantially 
focus o n a sign langu a g e v i deo displayed o n sa i d displa y mea ns 
wh en s aid deaf mute person views the outer world- — 

T-he^i^ee^--Rpu-t / o utput d e vieo -#eg-eeRvers at i o-n-w-i-t-h~s-ig-H- 

f ixirRg~-m eans has a f ra rae—s-t-r- u c t u3re whieh c a n be f ixed to t -h-e 
ear s a?*d— -Ro&e—o f a ^ea-^-isra-ke pep-sen-^ 



■8- -Th-e-vide© ■iHput/-©utput--deviee for-eonver satien-wi-th sign 

jraBr guag e — aeeer-d-i-ng — t-© — any — ©ne — ©-f — e-l-a-i-m-s — § — th-^eug-h — 7— 

langu age and 

s aid ta r get i m a- ging means is f ixed to sai d fixing mcan-s- 
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and picks up the area in th e v i ewing direction of th e de af mu t e 
person, — 



g_ A sign language int erp r e tation system intcrconnccti -ng- 

peg-sen capab-l-e of u-s-jrng— s-i-g-n lang u age-- fe-o-wh-are h is eeR-nee^-eeL 
%-he™-v^-dee— i-npu-t-/-eu%-pu-t — dev-i-e-e— fer — een-ve-r^a-t4-en— w-i-t-h— s-ig-n 

language — aeee-r^-i-ng— to — any — one — -©•# — e-l-aims — -5 — t-hr~e-ugh ~~8r — a 

v ideophon e — ter m inal — — n o n d eaf m ute — p erso ns — u-seel — by — a 
no n - dea f mut e pei^s or— -i-n capable o f u si -ng sign—jra-nguage-? — a-nd— a 
v i deophone t e rminal for sign lan g-ua-g c interpreters us ed by a 
sign language interp reter in order to p ro v ide sign la nguag e 
interpretation in a conversation between a d e af mute person 
and a non deaf mut e per s o n over a vide ophone, — char a cterized 
in that 

s-a-i-d-system includes terminale ompj^4-se-s connection means 

inc luding eq u ip ped wi t h a sign language interpreter 

registration table where the terminal number of the sa-id 

videophone terminal of_f •e^r-~s4-gn---i 

a sign language interpreter is registered; 

— said terminal connection means including a function 
to accept a call from said videophone terminal of said #e-p 
deaf-mute p erso npe rson -s or s-ai-d-videophone terminal of s ai dfo-p 
non-deaf-mute persons, a function to prompt a calling 
videophone terminal for which said call is accepted to enter 
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the terminal number of the called terminal, a function to 
extract the terminal number of the videophone terminal of a 
sign language interpreter from said sign language interpreter 
registration table, a function to call the videophone terminal 
of a#o-3r sign language interpreter int c^-pr^t-e^s by using said 
ext racted terminal numb e r ~-<>f-^hre~~si:g-n l ang^uage — -i-^-te^p-re-tre-ir , 
and a function to call the called videophone terminal by using 
said acquired called terminal number^ and 

video/ audio communications means including a function 
to synthesize at least a video from th e videophone terminal 
of said videophone t er minal f o -r—non- deaf-mute per son persons 
and a video from the said videophone terminal of said for sign 
language interpreter interpreters and transmit the resulting 
video to the^ said videophone terminal of said for deaf-mute 
persons-, a function to transmit at least a video from the sa id 
videophone terminal of said -£e-i^deaf -mute p e r s o np e-r-son-s- and 
an audio from the -s-a-i-d— videophone terminal of said #e-2r— sign 
language i n t e rpr e t er i-H-t-erpireter-s t o the said — videophone 
terminal for said non-deaf -mute p e r s o n pe- r-s-o n s->- and a function 
to transmit at least a video from the sa-i-d--videophone terminal 
of_said_#€Hr-deaf -mute persojipe^&ons- and an audio from the s aid- 
videophone terminal of said #ef-non- deaf -mute per sonpe rs o ns 
to th_e s-a-id— videophone terminal of sai d _-£e3r— sign language 
interpreters-. 
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74-0-. The sign language interpretation system according 

to the claim 6, wherein 9-, — characterized in that 

selection information for selecting a sign language 

interpreter is registered in said sign language interpreter 
registration table and -that said terminal connection means 
includes a function to acquire the conditions for selecting 
a sign language interpreter from said calling videophone 
terminal and a function to extract the terminal number of a 
sign language interpreter who satisfies said acquired 
selection conditions for the sign language interpreter from 
said sign language interpreter registration table. 

8 . The sign lang uage int erpretation system accor d ing 
to claim 6, wherein said _sign language interpretation^system 
includes a term registration table for registering a term used 
during sign language interpretation, wherein 
said terminal connection means includes a function to register 
a term in said term registration table via an operation from 
a videophone terminal, a function to select a term to be used 
from the terms registered in said term registration table via 
an operation from a videophone terminal, a function to generate 
a telop of said selected term, and a function to synthesize 
said generated^ telop onto a v ide o t o be transmitted to a 
conversation partner. 
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ABSTRACT OF THE DISCLOSURE 

A video input device for conversation using sign 

languagc videophonew i-t-h- sign language 7 — a— v ideo — a^ut-Mu-t^u^- 
device for conversation assistance device wir th sign languag e, 
a-nd — a — s-i-g-n — l-a^g-u-a-g-e — i-n- tcrpr ^e tation sys -tem— using — t-h-e — &ame 
e^p-atel^^-f—triran-sinirt-t-i-ng^^ 

pe-rfetfrmi^-^x-^^ rp.^. e . 

V"i"d'eo~i'n'pU""t~'de'Vire*e' f ■^3r"'eo*nver"S"a't~i"0'n~^ includes 
a target imaging camera (16) for imaging a target other than 
a sign language, a sign language imaging camera (1 4 ) for imaging 
the sign language of a deaf-mute person, a waist fixing device 
(15) for fixing the sign language imaging camera (1 4) at the 
waist of the deaf-mute person, a video synthesizer (17) for 
synthesizing the videos of the cameras — (44-? — 4-64-/ and a 

videophone connection device (18) for supplying the 

synthesized video to a videophone terminal — (4~©-)~. The 
input/output device for conversation — v-i-aw-i-t-h sign language 
further includes a display device ~(4~2-)~-f or displaying the sign 
language and a fixing device 44-34- f or fixing the display device 
in front of the eyes of the deaf-mute person, such so-that the 
sign language video being received at the video telephone 
terminal ( 1 0 )— is supplied to the display device. A— £4-2- ) . Th e 
sign language interpretation s y s t em s-ys-t-em — (- 100) provides a 
sign language interpretation service which can be used when 
a deaf-mute person converses with a non-deaf -mute person by 
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using the video — input / output above -described device — 
conversation with sign — language . 
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